Frustum VoxNet for 3D object detection from RGB-D or Depth images

Recently, there have been a plethora of classification and detection systems from RGB as well as 3D images. In this work, we describe a new 3D object detection system from an RGB-D or depth-only point cloud. Our system first detects objects in 2D (either RGB or pseudo-RGB constructed from depth). The next step is to detect 3D objects within the 3D frustums these 2D detections define. This is achieved by voxelizing parts of the frustums (since frustums can be really large), instead of using the whole frustums as done in earlier work. The main novelty of our system has to do with determining which parts (3D proposals) of the frustums to voxelize, thus allowing us to provide high resolution representations around the objects of interest. It also allows our system to have reduced memory requirements. These 3D proposals are fed to an efficient ResNet-based 3D Fully Convolutional Network (FCN). Our 3D detection system is fast and can be integrated into a robotics platform. With respect to systems that do not perform voxelization (such as PointNet), our methods can operate without the requirement of subsampling of the datasets. We have also introduced a pipelining approach that further improves the efficiency of our system. Results on SUN RGB-D dataset show that our system, which is based on a small network, can process 20 frames per second with comparable detection results to the state-of-the-art, achieving a 2 times speedup.Comment: page 8, add Acknowledgement. page 10, add Supplementary Material. The paper got accepted by 2020 Winter Conference on Applications of Computer Vision (WACV '20). The first arxiv version can be found here: arXiv:1910.0548

Integration of range and image sensing for photorealistic 3D modeling

The automated extraction of photorealistic 3-D models of the world that can be used in applications such as virtual reality, tele-presence, digital cinematography and urban planning, is the focus of this paper. The combination of range (dense depth estimates) and image sensing (color information) provides data-sets which allow us to create photorealistic models of high quality. The challenges are the simplification of the 3-D data set, the extraction of meaningful features in both the range and 2-D images and the fusion of those data-sets using the extracted features. We address all these challenges and provide results on data we gathered in outdoor scenes by a range and image sensor based on a mobile robot. Our ultimate goal is an autonomous 3-D model creation system which minimizes the amount of human interaction

Interactive sensor planning

This paper describes an interactive sensor planning system, that can be used to select viewpoints subject to camera visibility, field of view and task constraints. Application areas for this method include surveillance planning, safety monitoring, architectural site design planning, and automated site modeling. Given a description, of the sensor's characteristics, the objects in the 3-D scene, and the targets to be viewed, our algorithms compute the set of admissible view points that satisfy the constraints. The system first builds topologically correct solid models of the scene from a variety of data sources. Viewing targets are then selected, and visibility volumes and field of view cones are computed and intersected to create viewing volumes where cameras can be placed. The user can interactively manipulate the scene and select multiple target features to be viewed by a camera. The user can also select candidate viewpoints within this volume to synthesize views and verify the correctness of the planning system. We present experimental results for the planning system on an actual complex city model

Automatic registration of 2-D with 3-D imagery in urban environments

We are building a system that can automatically acquire 3D range scans and 2D images to build geometrically correct, texture mapped 3D models of urban environments. This paper deals with the problem of automatically registering the 3D range scans with images acquired at other times and with unknown camera calibration and location. The method involves the utilization of parallelism and orthogonality constraints that naturally exist in urban environments. We present results for building a texture mapped 3-D model of an urban building

3-D model construction using range and image data

This paper deals with the automated creation of geometric and photometric correct 3-D models of the world. Those models can be used for virtual reality, tele-presence, digital cinematography and urban planning applications. The combination of range (dense depth estimates) and image sensing (color information) provides data-sets which allow us to create geometrically correct, photorealistic models of high quality. The 3-D models are first built from range data using a volumetric set intersection method previously developed by us. Photometry can be mapped onto these models by registering features from both the 3-D and 2-D data sets. Range data segmentation algorithms have been developed to identify planar regions, determine linear features from planar intersections that can serve as features for registration with 2-D imagery lines, and reduce the overall complexity of the models. Results are shown for building models of large buildings on our campus using real data acquired from multiple sensors

From literary criticism to propaganda: intellectuals, culture, and politics during the Metaxas dictatorship (1936-1940)

This thesis discusses official discourse in conjunction with discourses produced by pro-regime intellectuals involved in cultural affairs and particularly literary criticism during the Metaxas dictatorship. Its primary sources consist chiefly of periodicals that supported the dictatorship or were published by it. The thesis attempts to place these discourses in a wider context by comparing them with those produced by far-right intellectual and political circles in other European countries in the first half of the twentieth century and by pinpointing their pre-1936 indigenous sources. It is argued that the level of erudition displayed by critics and cultural operators along with the regime’s self-styling as a Kulturstaat enabled such intellectuals to play a significant political (legitimising and propagandistic) role. The structure is thematic and revolves around these four themes: past, nation, authority and hierarchy, and future. The first topic is analysed through the discussion of specific historical periods that were the focus of Metaxist discourse and the modes of historical understanding that characterised it. The second theme is examined based on questions regarding the representation of the essence of Hellenism and the cultural definition of the nation, the emphasis on the homeland and its aesthetic or metaphysical attributes, the promotion of vernacular culture and language, and notions of national unity. The third topic encompasses propaganda related to order, discipline, and absolutism, as well as hierarchy, elitism, and the doctrine of charismatic leadership. The analysis of the fourth topic is pursued by means of discussing propaganda related to new chronotopes, the Third Hellenic Civilisation, and the youth

3-D modeling from range imagery: an incremental method with a planning component

In this paper we present a method for automatically constructing a CAD model of an unknown object from range images. The method is an incremental one that interleaves a sensing operation that acquires and merges information into the model with a planning phase to determine the next sensor position or "view". This is accomplished by integrating a system for 3-D model acquisition with a sensor planner. The model acquisition system provides facilities for range image acquisition, solid model construction and model merging: both mesh surface and solid representations are used to build a model of the range data from each view, which is then merged with the model built from previous sensing operations. The planning system utilizes the resulting incomplete model to plan the next sensing operation by finding a sensor viewpoint that will improve the fidelity of the model. Experimental results are presented for a complex part that includes polygonal faces, curved surfaces, and large self-occlusions

Automated model acquisition from range images with view planning

We present an incremental system that builds accurate CAD models of objects from multiple range images. Using a hybrid of surface mesh and volumetric representations, the system creates a "water-tight" 3D model at each step of the modeling process, allowing reasonable models to be built from a small number of views. We also present a method that can be used to plan the next view and reduce the number of scans needed to recover the object. Results are presented for the creation of 3D models of a computer game controller, a hip joint prosthesis, and a mechanical strut

Online Classification in 3D Urban Datasets Based on Hierarchical Detection

One of the most significant problems in the area of 3D range image processing is that of segmentation and classifi-cation from 3D laser range data, especially in real-time. In this work we introduce a novel multi-layer approach to the classification of 3D laser scan data. In particular, we build a hierarchical framework of online detection and identifica-tion procedures drawn from sequential analysis namely the CUSUM (Cumulative Sum) and SPRT (Sequential Proba-bility Ratio Test), both of which are low complexity algo-rithms. Each layer of algorithms builds upon the decisions made at the previous stage thus providing a robust frame-work of online decision making. In our new framework we are not only able to classify in coarse classes such as verti-cal, horizontal and/or vegetation but to also identify objects characterized by more subtle or gradual changes such as curbs or steps. Moreover, our new multi-layer approach combines information across scanlines and results in more accurate decision making. We perform experiments in com-plex urban scenes and provide quantitative results. 1

New methods for digital modeling of historic sites

We discuss new methods for building 3D models of historic sites. Our algorithm automatically computes pairwise registrations between individual scans, builds a topological graph, and places the scans in the same frame of reference